Effects of an Explicit Value Clarification Method With Computer-Tailored Advice on the Effectiveness of a Web-Based Smoking Cessation Decision Aid: Findings From a Randomized Controlled Trial

Introduction

Background

Smoking continues to be a major driver of global mortality [1], a trend that is mirrored in the Netherlands [2,3], which showcases that smoking prevention is imperative. However, as 21.7% of the adult Dutch population still smokes, according to recent figures [2], it is also of the utmost importance to invest in effective and evidence-based cessation assistance. Several cessation assistance tools are currently recognized as effective in aiding individuals in successfully achieving smoking abstinence. These range from behavioral interventions (eg, counseling [4]) to nicotine replacement therapy [5] and prescription medication [6]. Unfortunately, evidence-based cessation assistance is underused in various countries, including the Netherlands [7].

A common barrier to cessation assistance use is incorrect knowledge of the safety and efficacy of cessation assistance [8]. Therefore, increasing individuals’ knowledge may be a worthwhile avenue to explore when it comes to successfully increasing individuals’ cessation rates. In particular, being uninformed can lead to a state of uncertainty about which course of action to take—better known as decisional conflict [9]—which, in turn, is known to increase the chances of decision delay [9,10]. Therefore, individuals who decide to stop smoking but are uninformed about the possibilities of cessation assistance might delay the decision on how to stop smoking or simply decide to use the most common approach: attempting to quit smoking without the use of cessation assistance [11]. That said, knowledge provision alone is often insufficient to facilitate behavior change. This is commonly referred to as the knowledge-behavior gap in health promotion research [12,13]. Thus, knowledge provision alone might not be enough to support smokers motivated to quit smoking in their decisions about smoking cessation assistance. However, providing individuals who are motivated to stop smoking with decision support that also includes accurate information (next to other decision support elements) about cessation assistance tools might decrease decisional conflict. This, in turn, might facilitate smoking cessation efforts and ultimately increase the chances of long-term smoking abstinence.

Such decision support can be provided in the form of decision aids (DAs), which are interventions specifically designed to facilitate decisional processes [14]. A recent systematic review by Moyo et al [15] showed that DAs can be beneficial for smoking cessation, although traditionally, DAs have most often been applied to treatment and screening decisions rather than lifestyle-related decisions [14]. For example, BinDhim et al [16] showed that smoking cessation DAs can result in an increase in continuous abstinence at 1, 3, and 6 months compared with an intervention containing information only. Participants in the DA group were also more likely to have made an informed choice and showed fewer decisional conflict. However, information about the effective elements of smoking cessation DAs is currently lacking, and the only smoking cessation DA that has been previously studied in a Dutch context [17] has shown several limitations: it was largely paper based, thereby limiting widespread dissemination; it lacked an interactive design, although interactivity has been shown to positively influence factors such as information comprehensibility and attitudinal beliefs [18,19]; and it did not explicitly include methods of helping end users become aware of what is important to them personally (in the DA literature, this is often referred to as value clarification [20]), although this is regarded as an active DA element [21,22]. Moreover, interestingly, this DA had a positive effect on smoking cessation success but not on the uptake of cessation assistance [17]. Improving cessation assistance uptake might further increase the effectiveness of smoking cessation DAs, and overcoming the aforementioned limitations could play a promising role in achieving this.

To illustrate, given that a lack of knowledge is considered a barrier to cessation assistance use, adding interactive elements to a smoking cessation DA might be particularly helpful as interactive elements can improve information comprehensibility and positively influence individuals’ beliefs [18,19]. In addition to the use of interactive elements, explicitly devoting attention to smokers’ personal values could also positively influence the effects of smoking cessation DAs. The International Patient Decision Aid Standards Collaboration (IPDAS) regards these so-called value clarification methods (VCMs) as active DA elements [21,22]. VCMs can be divided into 2 different formats. Explicit VCMs refer to exercises that actively engage users in an activity to clarify what is important to them personally (eg, scoring certain statements), whereas implicit VCMs refer to the provision of static information that is specifically linked to the decision at hand. In other words, explicit VCMs include an element of interactivity that implicit VCMs lack. Recently, scholars have started to pay more attention to studying the added value of explicit (as opposed to implicit) VCMs. Previous studies have shown that explicit VCMs seem to be more effective than implicit VCMs in terms of decision-making processes [23], especially in the long run [24] and when people are supported in understanding the implications of their clarified values [25,26]. An approach to facilitate the understanding of the implications of clarified values is to show participants the options that best fit their clarified values [25]; for example, by providing computer-tailored advice based on answers provided in the explicit VCM. However, to date, it has not been studied whether the addition of explicit VCMs paired with such advice positively affects smoking cessation outcomes. To advance our understanding of the effectiveness of smoking cessation DAs and support more people in the Netherlands to quit smoking successfully, we developed a web-based smoking cessation DA (called VISOR) that includes interactivity and an explicit VCM paired with computer-tailored advice and studied its effects in a randomized controlled trial (RCT).

The Smoking Cessation DA VISOR

In accordance with the IPDAS guidelines for DA development [27,28], VISOR was developed by a steering team (TG, ESS, CDD, and CH) that led a development process involving both professional experts and potential end users, for example, by assessing their needs and opinions before the initial development [29] and by conducting usability tests—this development process is described in detail elsewhere [28].

Moreover, we used the self-determination theory (SDT) [30] as the theoretical background. The SDT revolves around the formation of motivation and posits that 3 psychological needs (ie, the needs for autonomy, relatedness, and competence) are essential to developing autonomous motivation (ie, motivation that emanates from oneself and is not primarily externally motivated) [30,31]. Autonomous motivation is assumed to have a greater influence on long-term behavior change compared with more controlled forms of motivation [32]. The SDT is particularly well-suited to support DA developers because it focuses on perceived autonomy as opposed to theories that are often used to develop more persuasive interventions. DAs are similarly geared toward personal autonomy. Therefore, the SDT supported us in developing a DA that not only provided accurate information but also helped motivate end users, for example, by framing the information in the DA autonomy supportively. Finally, VISOR took a stepwise approach to facilitate autonomous decision-making, which is concordant with personal values and smoking behavior. VISOR comprised 8 sections that are described in Textbox 1.

VISOR was a stand-alone, 1-time intervention meant to support adult smokers in the general population in their decision to use smoking cessation assistance and did not have to be used together with a health care professional. That said, VISOR could be used to prepare patients and clients for a health care consultation about smoking (cessation), and certain cessation assistance options (eg, prescription medication) required a health care provider to prescribe them. Therefore, if VISOR users chose to use a cessation assistance option that required a prescription from a health care provider, they were advised to contact their health care provider to gain access to this specific option. More in-depth information on VISOR, including the specific theoretical underpinnings of each step, can be found elsewhere [28]. Examples of the information section and explicit VCMs are shown in Figure 1 (information section [33]) and Figure 2 (explicit VCMs). The screenshots were translated from the original Dutch to English to facilitate understanding.

‎View this figure

‎View this figure

Study Goals and Hypotheses

In this study, the behavioral (ie, smoking abstinence and cessation assistance use) and decisional effects (ie, decisional conflict) of an explicit VCM paired with computer-tailored advice (within a smoking cessation DA) are reported. An RCT was conducted to investigate these effects. Specifically, we tested the following hypotheses (as described in the study protocol [28] and the Netherlands Trial Register [NL8270]):

• Hypothesis 1a or 1b is that a DA with explicit VCM and computer-tailored advice will lead to a statistically significant increase in smoking abstinence after 1 month (hypothesis 1a) and 6 months (hypothesis 1b) compared with a DA without explicit VCM and computer-tailored advice.
• Hypothesis 2a or 2b is that a DA with explicit VCM and computer-tailored advice will lead to a statistically significant increase in evidence-based cessation assistance use after 1 month (hypothesis 2a) and 6 months (hypothesis 2b) compared with a DA without explicit VCM and computer-tailored advice.
• Hypothesis 3 is that a DA with explicit VCM and computer-tailored advice will lead to a statistically significant decrease in decisional conflict (state of uncertainty about which course of action to take) immediately after using the DA compared with a DA without an explicit VCM and computer-tailored advice.

Methods

Overview

An RCT was conducted in line with the CONSORT-EHEALTH (Consolidated Standards of Reporting Trials of Electronic and Mobile Health Applications and Online Telehealth) checklist [34]. However, we deviated from the checklist in one aspect: participants were only invited for follow-up measurements if they completed one of the DAs; that is, if they completed the intervention until the end. This was because we wanted to ensure that the participants we included in the analysis received additional intervention elements as those were the focus of our RCT.

Ethics Approval

The study did not fall under the scope of the Medical Research Involving Human Subjects Act, as indicated by the Medical Ethics Committee of Zuyderland, the Netherlands (16-N-227; International Registered Report Identifier: RR2-10.2196/21772), and the development of VISOR and the accompanying studies (eg, the aforementioned needs assessment [29]) were funded by the Dutch Cancer Society (UM2015-7744). Study materials (including the underlying data and statistical scripts) can be found on the open science framework (OSF) website [35].

Primary and Secondary Outcomes

The primary outcome of this study was 7-day point prevalence after 6 months (corresponding to hypothesis 1b). Secondary outcomes were 7-day point prevalence after 1 month (corresponding to hypothesis 1a), evidence-based cessation assistance use after 1 and 6 months (corresponding to hypothesis 2a or 2b), and decisional conflict directly after using the DA (corresponding to hypothesis 3).

Sample Size

An a priori power calculation was conducted based on the primary outcome measure of 7-day point prevalence abstinence and the only other RCT in the Netherlands, which tested the effect of a smoking cessation DA in which a significant effect (20.2% vs 13.6%) was found at 6 months [17]. To be able to significantly (α=.05; β=.20) detect the same effect in a 1-sided test, 398 smokers per arm were necessary at the end of the trial (796 in total). Considering 50% attrition over the study period, we aimed to include 1592 smokers at baseline.

Study Population

Participants were included if they were (1) currently smoking, (2) motivated to stop smoking within 6 months, (3) aged between 18 and 100 years, (4) able to understand Dutch, and (5) had access to the internet and the necessary internet literacy (skills) to use the DA. The last 2 inclusion criteria were not actively screened but were deemed inherent to participation. Participants were excluded if they did not meet the inclusion criteria or exclusively used e-cigarettes. As described in the study protocol [28], participants were mainly recruited on the web to reflect the web-based nature of VISOR, and the entire trial was web-based (ie, there were no offline contacts). Recruitment took place mainly by using paid social media advertisements and unpaid social media posts on project accounts, [36] which were also shared on the team members’ accounts, their respective institutions, and other relevant organizations within the Netherlands. In addition, VISOR was featured in regional media (eg, a newspaper interview), and we used a project website with a direct access point to VISOR via a clickable button. Owing to the small influx of participants (especially after the start of the COVID-19 pandemic), we decided to deploy other additional recruitment activities as well, such as a study call, information in a relevant Dutch journal for general practitioners [37], and student pools at the Universities of Amsterdam and Maastricht. All (potential) participants received information about the content of VISOR, the duration of the study (including the number of follow-up measurements), and the compensation that they received for completing the last measurement (€10 [US $12.17]). Throughout the study, participants also received information on the duration of VISOR and the questionnaires. The participants received no information regarding the differences between the intervention and control groups to avoid bias in the results of the trial. The students at the University of Amsterdam received research credits instead of monetary compensation. All recruitment materials (eg, the project website) included a display of the project team’s institutional affiliations in some form.

Intervention and Comparator Groups

Participants in the intervention group received the DA as described in the Introduction section (see The Smoking Cessation DA VISOR section), whereas participants in the control group received the same DA, excluding the explicit VCM and computer-tailored advice; that is, steps 6 and 7 described in Textbox 1 were skipped. The only other (small) difference was that participants in the intervention group were immediately directed toward the end after they had chosen to not use evidence-based cessation assistance (ie, step 8—access information—was skipped), which was not the case for the control group. Thus, both groups had a chance to re-evaluate their choice, as the intervention group was offered a chance to re-evaluate their choice during the additional elements. Neither the DA received by the intervention group nor that received by the control group changed throughout the trial. Additional information can be found in the study protocol [28].

Trial Flow and Measurement Instruments

Overview

In total, the study comprised 4 fully automated and web-based contact moments: time point 0 for the baseline questionnaire and VISOR, time point 1 (t=1) directly after participants had used VISOR, time point 2 (t=2) after 1 month, and time point 3 (t=3) after 6 months. Participants were asked to fill in each follow-up questionnaire if they made use of the entire DA, even if they did not fill in one of the other follow-up questionnaires. To avoid high attrition rates, participants received either 1 automatic reminder after a week (if they had not filled in a follow-up questionnaire at all) or 2 after 2 days and a week (if they had already started filling in at least part of a follow-up questionnaire). Participants who started using VISOR or started filling in the baseline questionnaire (time point 0) without finishing it also received 2 automatic reminders (after 2 days and a week). In the last reminder for t=3, participants were also offered the option to share only their answers regarding the primary outcome (ie, 7-day point prevalence abstinence) with the research team. All data were self-assessed. If available, we used previously validated measurements [9] and measurements that were previously used in a Dutch context [38]; if possible, we used measurements that were previously used in self-administered web-based studies [39,40]. For more information, refer to the Checklist for Reporting Results of Internet E-Surveys checklist in Multimedia Appendix 1. In the case of psychological constructs collected using multiple items (eg, decisional conflict), we assessed scale quality, as proposed by Crutzen and Peters [41], using the Rosetta Stats package in R (R Foundation for Statistical Computing) [42,43] in two steps: (1) investigating scale structure by exploratory factor analysis and (2) calculating omega (Ω) [44] as a less biased alternative to (Cronbach) α.

Participants were registered for the study via a web-based form, which included their provision of informed consent and the creation of an account. Before account creation, participants were automatically randomized into either the intervention or the control group by the web-based platform on which questionnaires and VISOR were hosted, allocating approximately 50% of the respondents to either group. The end users were blinded to their allocated groups. Immediately after registration, the participants were asked to complete the baseline questionnaire. A visual representation of the trial flow can be found in the study protocol [28].

Baseline Measurements: Directly Before the DA

Demographic information was collected based on 3 criteria: age, gender, and education. Smoking behavior was collected regarding the used tobacco products, amount of tobacco consumption per product per day, past cessation attempts, amount of past cessation attempts (for people who previously attempted to stop smoking), and cessation assistance use in the past 6 months. If the participants indicated that they had used cessation assistance, they were also asked what had been used. Additional information regarding demographic information and smoking behavior can be found in Multimedia Appendix 2 [45-47].

Nicotine dependence was measured using the Revised Fagerström Test for Nicotine Dependence (FTND-R), which has shown better psychometric properties than the unrevised version [48] and was verified to be unidimensional within our sample (Ω=0.76; further information can be found on the OSF [35]). We changed the wording of the items slightly (eg, smoking moment instead of cigarette) to include other tobacco products as well and left out the item relating to the amount of cigarette consumption. Rather, we created a composite score based on participants’ answers to the item about the number of tobacco products, expressed as the number of cigarettes, with 1 hand-rolled cigarette and 1 other or a cannabis product equaling 1 cigarette, 1 pipe equaling 2.5 cigarettes, and 1 cigar equaling 4 cigarettes [38,49]. Subsequently, this composite score was recoded in line with the FTND-R (0=0-10 cigarettes, 1=11-20 cigarettes, 2=21-30 cigarettes, and 3=more than 30 cigarettes). We had to exclude e-cigarette use for this composite score as it was unclear how the answer categories of the e-cigarette use item related to the amount of cigarette consumption. Subsequently, a composite score was created by summing this item with the 5 other FTND-R items, resulting in 1 FTND-R score per individual ranging from 0 to 16, with 0 indicating no dependence.

Finally, the stage of decision-making was measured with 1 item: “Have you thought about how to quit smoking at this point? Choose the answer that best suits your situation; 1=I haven’t begun to think about the choices, 2=I haven’t begun to think about the choices, but am interested in doing so, 3=I am considering the options now, 4=I am close to selecting an option, 5=I have already made a decision, but am still willing to reconsider, 6=I have already made a decision and am unlikely to change my mind” [50].

Follow-up at t=1: Directly After the DA

After the DA, we measured decisional conflict (secondary outcome) using the Decisional Conflict Scale [9], which was also verified to be unidimensional in our sample (Ω=0.98; further information can be found on the OSF [35]). We used all 16 items using the original statement format with 5 response categories (0=strongly agree, 1=agree, 2=neither agree nor disagree, 3=disagree, and 4=strongly disagree) and created a composite score as described in the user manual provided by O’Connor [51]: individuals’ scores were summed, divided by 16, and multiplied by 25. Thus, every participant had a score ranging from 0 (no decisional conflict) to 100 (extremely high decisional conflict). Scores >37.5 were generally regarded as being associated with decision delay or being unsure about decision implementation [51].

Follow-ups at t=2 and t=3: 1 Month and 6 Months After Baseline

After 1 month and 6 months, participants were queried regarding the choice they made (ie, the implemented decision; secondary outcome) and whether they were able to abstain from smoking in the previous 7 days (ie, 7-day point prevalence abstinence; secondary and primary outcomes).

Data Analysis

All analyses were performed in R [43] using the integrated development environment of RStudio [52]. First, descriptive analyses were conducted to assess the sample characteristics.

Second, to determine which factors influenced nonuse attrition (ie, attrition during the intervention) and dropout attrition (ie, not returning to the follow-ups) [53], we first compared participants at baseline who did not finish VISOR with those who finished VISOR using the Wilcoxon rank-sum test, Mood median test, chi-square test, or Fisher exact test, depending on the variable in question. Subsequently, we checked the results of these univariate analyses by using logistic regression with nonuse attrition as the outcome and all significant variables from the univariate analyses as predictors. Regarding dropout attrition, we compared participants at t=1, t=2, and t=3 with those lost to follow-up (after having used the intervention) using the same approach as described previously.

Third, logistic regression was used to test hypotheses 1a, 1b, 2a, and 2b. First, we conducted crude analyses in which we only included the allocated group (ie, the intervention group compared with the control group) as a predictor and 7-day point prevalence abstinence as the outcome, followed by fully adjusted analyses in which we corrected for age, gender, education, and the FTND-R [28,54]. All covariates were selected a priori [28,54,55]; however, because of multicollinearity issues in some of the adjusted models testing hypotheses 2a and 2b, we needed to recode the educational variable into low and high rather than low, medium, and high (as for the other analyses). This was only done for the models where this was the case. In addition, because of the size of the 2 very small gender (identity) groups (ie, nonbinary participants and participants who preferred not to state their gender), we were unable to include participants belonging to these groups in all adjusted models. Therefore, we decided to include the data of those participants in the crude models but not in the adjusted models. To test the robustness of the results, logistic regression was conducted according to three different approaches: (1) complete cases only, (2) worst-case scenario (dropout respondents were considered to still smoke; ie, penalized imputation), and (3) multiple imputations (MIs) using the mice package [56]. Variables that were included in the imputation model can be found in Multimedia Appendix 3. In the MI models, we accounted for a technical mistake described in the Changes From the Study Protocol section. As we had directional hypotheses, we conducted 1-sided tests as planned a priori [28]. To calculate the P values, we used the following formulas: (1) P/2 if the effect moved in the hypothesized direction and (2) 1 – (P/2) if the effect moved in the other direction. Similarly, we swapped the upper bound of the calculated CIs to infinity. This was only done for the intervention effect as we had no specific directional hypotheses regarding the covariates.

Finally, hypothesis 3 was tested using linear regression. Again, we started with the crude analyses, using the allocated group as a predictor and the Decisional Conflict Scale as the outcome, followed by fully adjusted analyses corrected for covariates selected a priori (ie, age, gender, education, and stage of decision-making) [28,54,55]. Again, we had to exclude participants from certain analyses because they belonged to a very small gender (identity) group. Regarding hypothesis 3, we used only (1) complete cases and (2) MI using the mice package [56]. Variables that were included in the imputation model can be found in Multimedia Appendix 4. Again, we calculated the values for the 1-sided tests as described previously; however, as we expected a negative effect, we swapped the lower bound of the calculated CIs with infinity. In complete case analyses, all participants who filled in the respective outcome measures (eg, everyone who filled in the entire Decisional Conflict Scale) were included, even when participants did not finish the entire follow-up questionnaire.

Changes From the Study Protocol

In the original study protocol, and as described in the Netherlands Trial Register, participants were contacted 4 times after having used VISOR (ie, directly after the DA and after 1, 6, and 12 months). Unfortunately, we had to extend our recruitment period because of the COVID-19 pandemic, and consequently, the recruitment period lasted for approximately 12 months (ie, 6 months longer than initially planned). Consequently, given the maximum project duration funded by the Dutch Cancer Society, we did not perform the 12-month follow-up measurements. Therefore, the original primary outcome (ie, 7-day point prevalence after 12 months) had to be adjusted, and the 7-day point prevalence after 6 months was ultimately used as the primary outcome. This change was communicated to and approved by the Dutch Cancer Society. Other changes were not communicated and approved by them as this was not deemed necessary.

Furthermore, because of a technical mistake, some participants who completed VISOR but did not complete t=1 (directly after the DA) did not receive an automatic invite for the other follow-ups. When this mistake was discovered, some participants were already lost to follow-up. We dealt with this in two different ways: (1) people who had already missed t=2 but completed the DA in the 3 months before the discovery of the mistake received the invite to participate in t=2 regardless (38/599, 6.3% received this invitation and 3/599, 0.5% made use of this), and (2) people who had already missed t=3 were still invited to participate in t=3 but not t=2 (130/599, 21.7% received the invitation and 5/599, 0.8% made use of this).

Finally, we originally planned to adjust our analyses for covariates that were selected a priori if they were associated with outcomes. However, based on the advice of the involved statistician (SJ), we decided to adjust our analyses for all selected covariates a priori (as described in the study protocol [28]) to keep the covariates consistent across the different models and make the models comparable.

Results

Sample

The total sample comprised 2375 participants who were randomized, of whom 1164 (49.01%) completed the baseline questionnaire. Subsequently, of the 2375 participants, 599 (25.22%) completed one of the DAs, 276 (11.62%) filled in t=1 completely, 97 (4.08%) filled in t=2 completely, and 103 (4.34%) filled in t=3 completely. The entire trial flow is shown in Figure 3 [57]. The characteristics of the participants who completed the baseline questionnaire are shown in Table 1.

‎View this figure

Attrition

Nonuse Attrition

Comparisons of the participants who did not complete one of the DAs and those who did showed significant differences in group allocation (χ²₁ [N=1164]=15.2; P<.001), age (χ²₂ [N=1164]=78.3; P<.001), level of education χ²₂ [N=1164]=12.5; P=.002), whether people used other tobacco products (χ²₁ [N=1164]=4.6; P=.03), the average number of cessation attempts (W=121,312; P=.02), and stage of decision-making at baseline (W=153,400; P=.004). All comparisons can be found in Multimedia Appendix 5. In the logistic regression model, having been allocated to the intervention group (odds ratio [OR] 1.65, 95% CI 1.28-2.14; P<.001), being aged between 24 and 29 years (compared with 18-23 years; OR 0.60, 95% CI 0.38-0.93; P=.02), being aged between 30 and 100 years (compared with 18-23 years; OR 0.29, 95% CI 0.21-0.39; P<.001), and a high (compared with a low) level of education (OR 0.64, 95% CI 0.42-0.99; P=.046) remained significant; that is, participants in the intervention group were more likely not to complete VISOR, whereas those aged 24 to 100 years (compared with 18-23 years) and those with a high (compared with a low) level of education were more likely to complete VISOR.

Dropout Attrition

Comparisons of the participants who did not complete t=1 and those who did showed significant differences in group allocation (χ²₁ [N=599]=7.2; P=.01), gender (χ²₁ [N=598]=5.9; P=.02), and stage of decision-making (W=37,942; P=.001). All comparisons can be found in Multimedia Appendix 6. All variables remained significant in the logistic regression model: having been allocated to the intervention group (OR 0.65, 95% CI 0.47-0.90; P=.01), men compared with women (OR 0.65, 95% CI 0.46-0.92; P=.01), and stage of decision-making (OR 0.82, 95% CI 0.72-0.92; P=.001); that is, participants in the intervention group, men (compared with women) and those in higher stages of decision-making were less likely to drop out.

Comparisons of the participants who did not complete t=2 and those who did showed significant differences in group allocation (χ²₁ [N=599]=4.4; P=.04) and stage of decision-making (W=21,158; P=.03). All comparisons can be found in Multimedia Appendix 7. In the logistic regression model, having been allocated to the intervention group remained significant (OR 0.63, 95% CI 0.41-0.98; P=.04); that is, participants in the intervention group were less likely to drop out.

Comparisons of the participants who did not complete t=3 and those who did showed significant differences only in the stage of decision-making (W=21,601; P=.01), which also remained significant in the logistic regression (OR 0.84, 95% CI 0.72-0.97; P=.02); that is, participants in higher stages of decision-making were less likely to drop out. All comparisons can be found in Multimedia Appendix 8.

As next to the demographic variables (which were already planned as covariates for all analyses) and group allocation, only the stage of decision-making was most consistently associated with dropout; we decided to include the stage of decision-making as a covariate for all analyses as well—and not only decisional conflict as planned in the protocol [28].

Hypotheses Testing

Hypothesis 1a: Smoking Cessation After 1 Month

Although it was observed that more participants stopped smoking in the intervention group (15/55, 27% of the respondents) than in the control group (7/46, 15% of the respondents) after 1 month, the intervention did not result in a significant effect on smoking cessation in the complete case analyses (OR 2.09, 95% CI 0.79 to infinity [+], crude P=.07; OR 1.93, 95% CI 0.64 to infinity [+], adjusted P=.13) or the MI analyses (OR 1.26, 95% CI 0.63 to infinity [+], crude P=.25; OR 1.35, 95% CI 0.57 to infinity [+], adjusted P=.24). However, in the worst-case scenario, effects in favor of the intervention group were observed (OR 2.61, 95% CI 1.08 to infinity [+], crude P=.02; OR 2.71, 95% CI 1.09 to infinity [+], adjusted P=.02). In 2 of the 3 adjusted models (ie, complete cases and worst-case scenario), the only (other) variable that had a significant effect on smoking cessation was the stage of decision-making (complete case analysis: OR 1.83, 95% CI 1.24-2.85; P=.004; worst-case scenario: OR 1.63, 95% CI 1.22-2.22; P=.001). In the MI scenario, none of the included variables had a significant effect on smoking cessation rates after 1 month. Therefore, hypothesis 1a could only be confirmed in the worst-case scenario. Table 2 presents the results and more information.

Hypothesis 1b: Smoking Cessation After 6 Months

Although it was again observed that after 6 months, more participants reported having stopped smoking in the intervention group (23/63, 37% of the respondents) than in the control group (14/52, 27% of the respondents), the intervention did not result in a significant effect on smoking cessation in the complete case analyses (OR 1.56, 95% CI 0.71 to infinity [+], crude P=.14; OR 1.27, 95% CI 0.52 to infinity [+], adjusted P=.30) or the MI analyses (OR 1.35, 95% CI 0.59 to infinity [+], crude P=.23; OR 1.21, 95% CI 0.52 to infinity [+], adjusted P=.33). Similar to the results based on data collected after 1 month, effects in favor of the intervention group were observed in the worst-case scenario (OR 2.02, 95% CI 1.03 to infinity [+], crude P=.02; OR 1.85, 95% CI 0.92 to infinity [+], adjusted P=.04). In the worst-case scenario, the FTND-R (OR 0.87, 95% CI 0.78-0.98; P=.02; ie, nicotine dependence) and stage of decision-making (OR 1.37, 95% CI 1.09-1.72; P=.01) had a significant effect on smoking cessation rates. However, the FTND-R violated the linearity assumption in this model, but excluding it did not change the conclusions in relation to the primary outcome (Multimedia Appendix 9). In the complete case analyses and the MI scenario, none of the included variables had a significant effect on smoking cessation rates after 6 months. Therefore, hypothesis 1b could only be confirmed in the worst-case scenario. Table 2 presents the results and more information.

Hypothesis 2a: Use of Evidence-Based Cessation Assistance After 1 Month

Although, after 1 month, more people in the intervention group reported to have used an evidence-based cessation assistance tool (26/56, 46% of the respondents) than those in the control group (18/47, 38% of the respondents), the intervention did not result in a significant effect on the uptake of evidence-based cessation assistance in the complete case analysis (OR 1.40, 95% CI 0.64 to infinity [+], crude P=.20; OR 1.42, 95% CI 0.60 to infinity [+], adjusted P=.21) or the MI analyses (OR 1.25, 95% CI 0.69 to infinity [+], crude P=.23; OR 1.42, 95% CI 0.61 to infinity [+], adjusted P=.20). Similar to smoking cessation, effects in favor of the intervention group appeared in the worst-case scenario but only in the crude model (OR 1.78, 95% CI 0.96 to infinity [+], crude P=.04; OR 1.68, 95% CI 0.89 to infinity [+], adjusted P=.05). In the adjusted models, only the stage of decision-making had a significant effect on the outcome (in the complete case analysis, OR 1.53, 95% CI 1.11-2.19, P=.01; in the worst-case scenario, OR 1.36, 95% CI 1.10-1.68; P=.005; in the MI analysis, OR 1.47, 95% CI 1.07-2.02; P=.02). Therefore, hypothesis 2a could only be partially confirmed in the worst-case scenario. Table 3 presents the results and more information.

Hypothesis 2b: Use of Evidence-Based Cessation Assistance After 6 Months

Similarly, although, after 6 months, more people in the intervention group reported using an evidence-based cessation assistance tool (38/61, 62% of the respondents) than those in the control group (26/50, 52% of the respondents), the intervention did not result in a significant effect on the uptake of evidence-based cessation assistance in the complete case analysis (OR 1.53, 95% CI 0.72 to infinity [+], crude P=.14; OR 1.71, 95% CI 0.74 to infinity [+], adjusted P=.11) or the MI analysis (OR 1.72, 95% CI 0.83 to infinity [+], crude P=.07; OR 1.65, 95% CI 0.69 to infinity [+], adjusted P=.13). However, in the worst-case scenario, effects in favor of the intervention group emerged (OR 1.84, 95% CI 1.09 to infinity [+], crude P=.01; OR 1.80, 95% CI 1.06 to infinity [+], adjusted P=.02). In the adjusted models, the stage of decision-making had a significant effect on the outcome but only in the worst-case scenario (OR 1.27, 95% CI 1.06-1.53; P=.01). Therefore, hypothesis 2b could only be confirmed in the worst-case scenario. Table 3 presents the results and more information.

Hypothesis 3: Decisional Conflict Immediately After the DA

Despite the small difference in averages between the intervention (mean 39.29, SD 25.00) and control groups (mean 41.17, SD 25.23), the intervention had no significant effect on the decisional conflict in either the complete case analysis (β=−0.04, crude P=.25; β=−0.05, adjusted P=.20) or MI analysis (β=−0.05, crude P=.18; β=−0.05, adjusted P=.22). In the adjusted models, only the stage of decision-making (β=−0.15, P=.005 in the complete case analysis; β=−0.14, P=.01 in the MI analysis), being aged between 30 and 100 years (compared with 18-23 years) (only in the MI analysis, β=−0.06, P=.045), and a high level of education (β=−0.27, P=.002 in the complete case analysis; β=−0.25, P=.005 in the MI analysis) had a significant effect on the outcome. All of them had a negative effect; that is, people who reported a higher stage of decision-making, participants aged between 30 and 100 years old (compared with 18-23 years), and participants with a high level of education (compared with participants with a low level of education) experienced less decisional conflict. Therefore, hypothesis 3 could not be confirmed. Table 4 presents the results and more information.

Discussion

Principal Findings

The aim of this paper was to report the effects of adding an explicit VCM with computer-tailored advice to a smoking cessation DA (VISOR) on both smoking cessation outcomes and decisional conflict. Contrary to our expectations, we did not find any effect on decisional conflict. In addition, although the worst-case scenarios might suggest an effect on smoking cessation rates and cessation assistance uptake, this finding was not replicated in either the complete case analyses or MI analyses. Moreover, given the fact that Blankers et al [58] showed that analyses based on penalized imputation can be biased when missingness is unbalanced between trial arms (as in our case), we cannot confidently speak of the effects on smoking cessation success and cessation assistance uptake, despite the suggestion of effects in the worst-case scenarios. That said, all the significant and nonsignificant effects that were found were in the expected direction; that is, participants in the intervention group showed more smoking cessation, more evidence-based cessation assistance uptake, and less decisional conflict. A summary of the main findings with respect to the primary and secondary outcomes can be found in Textbox 2.

However, based on these findings, it may be too early to declare the addition of explicit VCMs and computer-tailored advice as ineffective to smoking cessation DAs. Previous studies have consistently found effects in favor of explicit VCMs on value-congruent decision-making in other contexts, especially when combined with computer-tailored advice [22,25]. That said, we were the first (to the best of our knowledge) to test this in a smoking cessation context. We would also like to emphasize that our findings do not imply that smoking cessation DAs are ineffective, as we did not compare a smoking cessation DA with usual care, another intervention, or no intervention but rather compared 2 different versions of one and the same DA: one with a VCM and paired computer-tailored advice and one without. In fact, even the control group in this RCT exceeded the smoking cessation rates achieved by the only other Dutch smoking cessation DA described in the literature [17]. However, this study showed that adding explicit VCMs paired with computer-tailored advice might not always be a good idea, as nonuse attrition was 34.19% higher in the intervention group than in the control group. In addition, our nonsignificant findings entail some caveats. In this discussion, we will focus on the two most likely explanations for the lack of effects we found: (1) we were unable to find an effect because of a lack of power, and (2) there truly was no effect. In addition, we also describe the implications of these 2 explanations and how readers might benefit from the insights generated during our project.

Lack of Statistical Power

The most likely explanation for the lack of significant effects was the lack of statistical power. During this trial, we faced a relatively high attrition rate (as is typical for digital interventions [53]), and ultimately, our trial was widely underpowered as even the MI analyses were underpowered to conduct our analyses. On the basis of our power analyses, we planned to include 1592 smokers at baseline, accounting for 50% of attrition. Ultimately, we randomized 2375 participants but had already lost 1211 (50.99%) individuals from this group between randomization and the end of the baseline questionnaire. Within this group, we subsequently lost 48.54% (565/1164) of participants to nonuse attrition and, among the participants invited to follow-up, 53.92% to 83.8% of participants to drop out attrition. Therefore, attrition was much higher than originally anticipated. Although replication of our study with a larger sample is recommended, researchers and DA developers can learn from the reasons for attrition uncovered in this project (eg, use time), as will be explained in the following sections.

Of the allocated participants, only 23.63% (275/1164) of the participants allocated to the intervention group completed VISOR compared with 34.29% (324/945) of the participants allocated to the control group. Interestingly, the available data provided by the intervention host indicated that the differences in use times between the 2 groups were significantly different. The median use time of the intervention group was approximately 9 minutes, whereas that of the control group was approximately 6 minutes (see the OSF for more information [35]). This difference may have driven the differences in nonuse attrition between the intervention and control groups. Our other findings regarding nonuse attrition also give credence to this explanation as younger participants were more likely to not complete VISOR. During the needs assessment conducted to develop VISOR, younger participants in particular indicated that relatively short time frames were acceptable for using a smoking cessation DA [29]. Therefore, future research should explore whether VCMs and computer-tailored advice can be designed to be delivered in a shorter manner to alleviate this problem. For example, Witteman et al [25] reported a digital VCM (including computer-tailored advice) that comprised dynamic web sliders representing both values and preferences (ie, the included options). These web sliders were then linked, meaning that for participants who indicated that a particular value was very important to them, the slider representing the preference moved equally. Such a VCM could potentially be much shorter to use as end users do not have to answer multiple statements. However, at this point, this technique is relatively difficult to use for decisions involving multiple options, which is why we were unable to use it within VISOR. However, on the basis of our findings, one might also conclude that it is important to spend more time studying user experience components, especially use times. A well-known method of studying this is the think-aloud method (which has also been used to test VISOR’s usability [28]). In studies using the think-aloud method, participants are asked to use an intervention while verbalizing their thoughts to uncover the cognitive processes and emotional reactions when using the intervention [59,60].

Interestingly, based on our data, it can also be concluded that we did not experience a recruitment problem (ie, 2109 participants were initially randomized and eligible to participate) but rather a problem of retention—only a small number of participants completed VISOR and even fewer participants completed the follow-up questionnaires. Although this is often observed in digital health care interventions [61], it is crucial to find ways of increasing the actual use of digital DAs and retention in DA trials to ensure benefits for users and also reach sufficient statistical power during studies. A way of achieving this would be to embed DAs (such as VISOR) in a counseling pathway as the involvement of a professional has been shown to positively influence time spent on websites aimed at improving healthy lifestyles [61]. Concurrently, this might also positively influence retention rates in studies (ie, people returning for follow-up measurements) in which digital DAs are evaluated [62,63]. There could be multiple ways of accomplishing this in the context of VISOR (or other digital DAs for that matter): (1) VISOR could be used together with a health care professional (eg, a practice nurse) or could be sent to participants before or after a health care consultation in which (the result of) VISOR is discussed, or (2) a digital form of counseling could be included in VISOR (eg, through the form of videocalls [64] or an automated chatbot [65]). The second approach might be especially promising as it would keep the fully digital and automated nature of VISOR intact, thereby still ensuring the optimal reach of the DA. Future research should investigate which modalities are especially beneficial in terms of outcomes and retention and which modalities are preferred by end users themselves.

Related to the problem of retention, an interesting chance finding of our study was that it highlighted the importance of the stage of decision-making in this regard, as this influenced both nonuse and dropout attrition. Interestingly, this influence seemed to disappear (partially) once we corrected for other dropout predictors. The most plausible explanation for this is that different groups within our sample (eg, older compared with younger participants) differed in their stage of decision-making (see the OSF for more information [35]), leading to the stage of decision-making becoming insignificant once all predictors were added in the same model. Interestingly, the stage of decision-making is much less routinely assessed than decisional conflict; to illustrate, the validation article of the original Decisional Conflict Scale [9] has been cited >2000 times according to Google Scholar, whereas the user manual of the stage of decision-making scale [50] has only been cited a little over 30 times and, when it is assessed, it is mostly done to include it as a covariate [66,67]. Consequently, the roles of these decisional stages seem to be much less understood. That said, it is assumed that individuals’ stages of decision-making influence their receptiveness to DAs and that people who are in active stages of deliberation would benefit the most from DAs, whereas people who either have not even begun to think about the decision or are unwilling to reconsider their decision may not benefit as much from a DA [50]. Interestingly, the largest group (approximately 40%) in our sample was at an early stage of decision-making, which is also reflected in the respondents’ average score for this variable (ie, 2.92, SD 1.4), which is slightly below active deliberation (ie, the stage in which individuals start weighing the different options). As such, it might be the case that VISOR (with or without the explicit VCM and advice) simply overwhelmed a big part of our sample. The relatively high decisional conflict scores in both groups seem to confirm this. Therefore, tailoring DAs to the stage of decision-making might be a promising approach to further limiting attrition. Although demographic variables seem to be the more obvious choice based on our data, the 1-item scale used to assess the stage of decision-making has the major advantage of being easy to use and not requiring participants to complete lengthy questionnaires. By tailoring DAs to the stage of decision-making (as opposed to other constructs or demographic characteristics), it would be possible to alleviate one of the core problems of most contemporary approaches to computer tailoring—that they often impose a significant burden on participants [68]. The fact that participants’ stage of decision-making had the most consistent effect on the outcomes among the included predictors only adds to this. For example, individuals in the initial stages could be offered interventions that are merely educational and supportive in nature, whereas individuals in active deliberation stages could be offered traditional DAs, and individuals in later stages could be offered support to implement their decision. To illustrate, individuals in an early decision-making stage could be offered a brief intervention aimed at increasing motivation to stop smoking or use evidence-based cessation assistance (similar to the 5 As [69]). Once they have reached a higher stage, they could then be provided with a traditional DA, such as VISOR. In other words, insights from behavior change could be used to further improve DAs for specific groups of decision makers [70].

True Absence of an Effect

Owing to the lack of statistical power, we cannot confidently conclude that there truly was no effect of VISOR. However, interestingly, Sheridan et al [71] also tested the additional effect of an explicit VCM added to a DA aimed at a prevention-related decision (ie, heart disease prevention) and found no effect on outcomes such as decisional conflict or intention to reduce heart disease risk. However, most other studies tested DAs (and by extension explicit VCMs) in a treatment context (eg, between different surgeries [25]), with positive effects reported quite often. On the basis of this contradiction, we could deduce that prevention-related decisions are somewhat different and may not be affected by DAs in a manner similar to treatment decisions. Anecdotally, throughout the project in which VISOR was developed and tested, people not involved in the project often stated that prevention-related decisions (such as smoking cessation decisions) seemed easier to make than other decisions, such as treatment decisions. Although this conclusion is logical to a certain degree, as treatment decisions often involve much more imminent risks, such as death in the near future [25], we found no indications that the participants in this study regarded the decision among different cessation aids as easy. In fact, participants in both trial arms experienced decisional conflict that scored above the accepted cutoff value of 37.5 [51]. The decisional conflict scores observed in this study were also higher than those reported in the literature [72], which tend to focus on treatment decisions, indicating that prevention-related decisions are not necessarily easier to make than treatment decisions. In other words, there is no indication that prevention-related decisions are perceived differently from treatment decisions (at least not in terms of experienced difficulty), meaning that they can be assumed to respond similarly to intervention elements commonly included in DAs (such as VCMs). However, based on our data, it cannot be fully excluded that prevention-related decisions differ in factors other than experienced difficulty and therefore do not respond to explicit VCMs as expected. A more viable difference between prevention-related DAs and DAs focused on treatment decisions is that those focused on prevention are often used without the direct involvement of a health care professional [16,17]; that is, prevention-related DAs are more likely to be self-administered [73] than DAs focused on treatments. However, a recent systematic review and meta-analysis by Larsen et al [73] showed that self-administered DAs for colorectal cancer screening can also be beneficial in relation to prevention-related decisions, such as population screening, showcasing that this explanation (ie, lack of health care involvement) is unlikely. In other words, at this point, no convincing argument can be made about prevention-related decisions being different from other health care decisions. Therefore, they should respond similarly to the elements typically used in DAs. However, given the substantial amount of heterogeneity in the literature on DAs aimed at primary prevention [74], it may be worth investigating further whether prevention-related decisions really respond to DA elements, as one would expect based on the wider DA literature; for example, by replicating our work with a larger sample. In addition, more research could be conducted on perceived differences between prevention-related decisions and other health-related decisions (eg, treatment decisions); for example, by conducting in-depth interviews with participants who recently took both to discover perceived differences in their decision-making.

Strengths and Limitations

The major strength of our study was that it was conducted in a real-life context, and we mainly recruited individuals who were interested in smoking cessation (as opposed to the hypothetical scenario often used to evaluate VCMs [25]). In addition, our sample was relatively representative of the Dutch population, except for women being overrepresented and participants with a low level of education being slightly underrepresented [75]. Thus, our findings might be largely generalizable to the smoking Dutch population with access to the internet. The fact that both groups received a DA also allowed us to blind participants, which strengthened the RCT, as participants were not biased in the sense that they knew to which intervention arm they were allocated and, thus, what the other group was offered. Finally, we focused our analyses on 3 outcomes of interest, thereby decreasing the risk of a type I error.

However, this study also has some limitations. Owing to our study design, we might have experienced a selection bias as we only invited people to complete the follow-ups if they had used the entire intervention. We tried to alleviate the influence of this selection bias by including variables associated with this selection (eg, age) as covariates in the analysis and imputation models. However, it is possible that more motivated smokers completed the intervention and therefore, for example, showed higher quit rates. In addition, we had to change the primary outcome and shorten the follow-up period to 6 months after baseline. Although unfortunate, we do not consider this to be a major issue as experts regard a follow-up period of 6 months to be sufficient, as most relapses occur relatively soon after the quit attempt [76-78]. In addition, the trial showed such large attrition that we were underpowered for most of our analyses. That said, the findings from the complete case analysis and those based on MI led to the same conclusions, making our findings more robust. It might also be a limitation that we relied on self-reported data. However, as previous research has shown that self-reported smoking status tends to be accurate [79] and because participants received the reward for study completion regardless of their smoking status, we are confident that this has had little to no influence on our findings. Finally, we were unable to separately test the effects of explicit VCM and computer-tailored advice. This would have required 3 trial arms and an even larger sample size and was therefore deemed unfeasible. It should also be recognized that digital, web-based DAs, by definition, exclude people who either do not have access to the internet at all or do not have the necessary digital literacy to use it. Although we consider this limitation to be small in the Dutch context, given the large number of Dutch households with internet access [80], it is certainly important to consider this to achieve full health equity. In the same vein, it is also important to consider other characteristics (eg, health literacy and financial resources to access smoking cessation assistance) that may influence the ability of end users to benefit from smoking cessation DAs. For example, the findings of our attrition analyses show that future projects should place a stronger emphasis on end users with less formal education. Finally, this study focused on direct behavioral and decisional effects only, as indirect effects were beyond the scope of this study. The same applies to cost-effectiveness and cost-utility. Investigating indirect effects (as hypothesized in the study protocol [28]), cost-effectiveness, and cost-utility might provide additional insights, in addition to the direct effects reported in this study.

Conclusions

At this point, we cannot confidently recommend the inclusion of explicit VCMs and computer-tailored advice in smoking cessation DAs. In fact, these 2 elements might result in higher attrition rates during the use of DAs, thereby limiting their potential. However, our findings in relation to the primary and secondary outcomes might be influenced by a lack of statistical power; therefore, we advocate for the replication of our trial with a larger sample. Finally, researchers should emphasize user experience, especially use times, and experiment with (innovative) solutions to deal with a (too) high perceived user burden, such as the integration of VISOR within a counseling pathway or tailoring VISOR and other DAs to people’s decision-making stages. Finally, researchers should investigate how prevention-related decisions respond to DA elements. Given the strong initial interest in VISOR and the potential of smoking cessation DAs to combat tobacco-related mortality by inducing more smoking cessation attempts [15], future research should continue to find ways of optimizing smoking cessation DAs for future use.